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ABSTRACT: Since decades, face recognition has become an active area of research in computer vision and psychology. 
The rapid developments of face recognition are being fueled by numerous advances in computer vision. An ongoing 
challenge in this field is to design an effective human-computer interaction (HCII). In this paper we will study the latest 
work done that has been dime in the field of facial expression recognition and analysis. In our work we have recognized six 
different expressions using Cohn-kanade database and system is trained using scaled conjugate gradient back-propagation 
algorithm. In proposed methodology we have used MAT LAB' s computer vision toolbox for face detection & cropping the 
images and neural network toolbox. In our work we have achieved 100% training accuracy and 87.2% overall testing 
accuracy of six different expressions. 

Keywords: FACS (Face action coding system), neural network, Action units (AU), SVM (Support vector machines). 
I. INTRODUCTION 

Human-computer interaction such as voice, gesture, and force-feedback are emerging. Despite important advances, 
one necessary ingredient for natural interaction is still missing that is emotions. Emotions play an important role in human- 
to-human communication and interaction, allowing people to express themselves beyond the verbal domain. The ability to 
understand human emotions is desirable for the computer in several applications. The facial expressions are one of the most 
powerful channels of non-verbal Communication. Facial expression provides information about emotional response, 
regulates interpersonal behavior, and communicates aspects of psychopathology. Facial expressions can reveal what people 
are thinking and feeling, it is only recently that the face has been studied scientifically and has the great potential for human- 
computer interaction. 

Several approaches have been used for automatic facial emotions recognition from static images or video 
sequences. In all these approaches, the first step is to detect the face and once the face is detected the next step is to extract 
the features from the detected face that are relevant to display of emotions and classified into a predefined set of facial 
actions or furthermore to emotion related expressions. Most of the facial emotion or expression analyzers recognize 
expressions corresponding to the basic emotions, i.e happiness, anger, fear, surprise, disgust and sadness. Fig. 1 explains 
basic steps that are used for facial expression analysis system. 
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Fig 1:- Basic Face expression analysis system 



First step is the input image with different expressions play an important role in the facial expression analysis. 
Properties of the image like its resolution, sizes etc are important and usually, the facial image in the frontal or near frontal 
view is used to recognize facial expression. Once we have the image the next step is to cut and crop the image and detect 
face using the face detector. Now after second step we got the balanced image whose feature should be fed to the system. 
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The facial features are mainly of two types: geometric and appearances features [3]. Geometric features measures the 
variation in shape, location, distance of facial components in different expressions. The appearance features can be extracted 
on either whole face or specific regions in a facial image. The last and most critical step is the expression classification. The 
classification systems classify the emotions into different categories according to the mood. 

In the next section we will study about the related work that has been done in the field of face expression 
recognition and analysis. 

II. Literature review 

Research on human perception and cognition has been conducted for many years, but it is still unclear how humans 
recognize facial expressions. Which types of parameters are used by humans, and how are they processed? By comparing 
human and automatic facial expression recognition we may be able advance our understanding of each and discover new 
ways of improving automatic facial expression recognition. Although it is often assumed that more fine-grained recognition 
is preferable, the answer depends on both the quality of the face images and the type of application. Ideally, an automatic 
face expression system should recognize all action units and their combinations. In high quality images, this goal seems 
achievable; emotion-specified expressions then can be identified based on emotion prototypes identified in the psychology 
literature. For each emotion, prototypic action units have been identified. In lower quality image data, only a subset of action 
units and emotion specified expression may be recognized. Recognition of emotion-specified expressions directly may be 
needed. We seek systems that become 'self-aware' about the degree of recognition that is possible based on the information 
of given images and adjust processing and outputs accordingly. Recognition from coarse-to-fine, for example from emotion- 
specified expressions to subtle action units, depends on image quality and the type of application. 

Two main streams in the current research on automatic analysis of facial expressions consider facial effect 
(emotion) interference from facial expressions and facial muscle action detection [1]. In this section, we also evaluate the 
various frameworks for emotion detection. The objective is to assess the relevance of different framework to deal with a 
different kind of data. 

2.1 Facial Action Coding System (FACS) 

The Facial Action Coding System (FACS) is a comprehensive, anatomically based system for measuring nearly all 
visually discernible facial movements [4]. FACS describes facial activity on the basis of action units (AU), as well as several 
categories of head and eye positions and movements. Action Units (AU) are the fundamental actions of individual muscles 
or groups of muscles. FACS is recognized as the most comprehensive and objective means for measuring facial movement 
currently available, and it has become the standard for facial measurement in behavioral research in psychology and related 
fields. Since FACS deals with the movement, not with other visible facial signs, it limits a full understanding of the 
psychology of facial behavior. Thus the person performing the classification has to be trained to interpret the expression 
from the action units obtained. 

2.2 Neural network based analysis 

Neural network learning methods provide a robust approach to approximating real -valued, discrete-valued, and 
vector-valued target functions. For certain types of problems such as learning to interpret complex real-world sensor data, 
artificial neural networks are among the most effective learning methods currently known. For example, the 
BACKPROPAGATION has proven surprisingly successful in many practical problems such as learning to recognize 
handwritten characters (Lecunn et al. 1989), learning to recognize spoken words (Lang et al 1990) and learning to recognize 
faces (CottreU 1990). 

Padgent [5], Hara and Kobayashi [6, 7], Zhang [8] and Zhao [7] used neural network approach for expression 
classification. They classified images into six or seven emotional categories. Padgett et al., [5] trained neural networks from 
the data of 1 1 subjects and tested with the data from one subject. The training and testing dataset was interchanged and new 
networks were trained and tested. A classification accuracy of 86% was achieved in this study. Hara and Kobayashi [6, 7] 
also used neural networks approach. The training dataset consisted of from data of 15 subjects (90 images) and these 
networks were tested using data from another 15 subjects. The classification accuracy achieved was 85 %. Zhang et al., [8] 
used the JAFFE data base which consists of 10 Japanese female subjects. Although an accuracy of 90.1% was achieved; 
same data was used for training and testing. 

2.3 EMG based methods 

Facial EMG measures the electrical activity of the facial muscles [9]. Facial expression analysis using EMG based 
techniques requires invasive insertion of electrodes into the facial muscle fiber for accurate result. The major disadvantage of 
using EMG based methods is that it may alter the normal behavior of the subjects due to attachment of electrodes and 
confuse the subject. 

Various teams have worked on the face expression recognition and analysis systems using FACS AU. 
The results analysis work is done in two categories: 

A) Person Independent:- Person independent category means in such systems, during training few images are not shown to 
the system and we will use those images to check how many images the system identify based on the mood correctly. 

B) Person Specific: - Person specific category means the faces that we have trained or shown to the system and testing is 
done on the same faces. 
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Teams 


Emotion Detection 


Overall 


Person 
Independent 


Person 
Specific 


ANU [16] 


0.649 


0.838 


0.734 


KIT [1] 


0.658 


0.944 


0.773 


MIT- 
Cambridge 
[1] 


0.448 


0.433 


0.44 


Montreal 
[1] 


0.579 


0.870 


0.700 


NUS [21] 


0.636 


0.730 


0.672 


Riverside 
[2] 


0.752 


0.962 


0.838 



ISSN: 2249-6645 



Table 2:- Emotion recognition using SVM and their accuracies by different teams. 

III. Experimentation 

In this section we will explain the implementation of our proposed work. Our proposed work is divided into four main 
steps explained below: 

Step 1 .--Collection of different face expression images 

First step is to collect different images of different facial expressions. So for that we have used cohn-kanade database. It 

consists of six basic emotions that we need to classify. 

Step 2:- Creation of dataset 

Step 3:- Selection of neural network 

Once the dataset is created the next step is to select the neural network. It means we need to decide about input layer, hidden 
layer and output layer. For our work, we have used 50*50 images it means our input layer consist of 2500 features. We have 
used 200 hidden layer and since we want to classify six different emotions hence our output layer consist of 6 classes. 



.1 



Fig: Neural network used 
Step 4:- Training the system using Scale conjugate gradient back propagation algorithm. 

Training set 



learning Algorithm 



Output 



Once the network architecture is decided the next step is to train this network. For that we have used scale conjugate gradient 
back-propagation as learning algorithm 

Step 5: Testing the system using unseen images. 

Once the system is trained, Test input i.e. unseen images that are not shown to the system is provided to the system and 
according to hypothesis the image is classified accordingly. 
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IV. Results 

In this section, we will evaluate the proposed method. We have implemented our face recognition system using MATLAB. 
We have used Cohn-Kanade database for emotion recognition which contains six different emotions. Classification rates [1] 
by the baseline method for the emotion detection sub-challenge are shown in Table V. After simulating, we have got good 
training and testing accuracies as shown in table 3. 



Training accuracy 


100% 


Testing Accuracy 


87.2% 



Table 3: - Overall Training and testing accuracy 



We have also measure performance of our system by estimating mean square error and % Error. 

Mean squared error is the average squared difference between outputs and targets. Lower values are better. Zero 
means no error. 

Percent Error indicates the fraction of samples which are misclassified. A value of 0 means no misclassification, 
100 indicates maximum misclassification. 





Mean square error 


Error(%) 


Training 


1.16455e-5 


0 


Testing 


4.04047e-2 


12.76595e-0 



Table 4:- Mean square error and Error (%) 



In addition, we have provided confusion matrices for the for emotion recognition of the overall test dataset. Rows are 
predicted results, columns the ground truth. 





Angry 


Disgust 


Fear 


Happy 


Sad 


Surprise 


Angry 


3 


0 


0 


0 


0 


0 


Disgust 


0 


5 


1 


0 


0 


0 


Fear 


0 


0 


7 


0 


1 


0 


Happy 


0 


2 


0 


12 


0 


0 


Sad 


2 


2 


1 


0 


5 


0 


Surprise 


0 


0 


0 


0 


0 


9 



Table 5:- Confusion Matrix for emotion recognition of the overall test dataset. 

V. Conclusion 

This paper describes the different techniques that are employed in face expression recognition and analysis. With 
respect to machine learning techniques, we noticed a strong trend to use SVMs. Most of the teams result that we have 
already shown in Table 2 used SVM, such techniques have proven very popular in recent literature. But in our work we 
have used scale conjugate gradient back propagation algorithm and we are getting overall testing accuracy up to 87.2% 
which is better than the as compared to the work done using SVM explained in table 2. 

In our future work, we can improve our recognition rate by using LBP histogram, equalization techniques & PCA 
techniques before training the system. 
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